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SUMMARY 


The  Idea  of  *  stationary  distribution  for  a  Markov 

*  ^ 

process  can  bs  extended  to  Include  distribution^  or  measures 
which  are  infinite.  It  is  shown  that  a  certain  type  of  recur¬ 
rence  condition  implies  the  existence  of  a  possibly  infinite 
stationary  measure.  Applications  are  discussed 
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THE  EXISTENCE  OF  STATIONARY  MEASURES  FOR  CERTAIN 

MARKCV  PROCESSES 


1 .  Introduction 

We  consider  a  Markov  process  xn ,  n  -  0,1,....  The  random 
variables  x^  belong  to  an  abstract  set  X  In  which  a  Borel  field 
B  Is  defined,  X  Itself  being  an  element  of  B.  It  is  assumed 
tnroughcut  tnls  paper  that  B  Is  separable;  that  le  B  1b  the 
Borel  extensl  n  cf  a  denumerable  family  of  sets.  The  transition 
law  of  the  process  Is  given  by  a  function  F(x,E)  -  P*(x,E), 
this  function  being  interpreted  as  the  conditional  probability 
that  x^tl  t  L,  given  x^  -  x.  Tne  n-step  transition  probability 
Is  designated  by  Pn(x,E).  When  conditional  probabilities  are 
used  beltw,  It  will  usually  be  understood  that  they  are  the 
ones  uniquely  determined  by  the  transition  probabilities. 

Tne  sets  In  B  will  sometimes  be  called  "measurable  sets". 

Tnroughcut  this  paper  a  "measure"  ^111  mean  a  countably 
additive  set  functl  n,  defined  v  n  the  measurable  sets,  non- 
negative,  ana  n  t  Identically  0.  (Tne  w^rds  "countably  additive" 
will  sometimes  ce  repeated  f  r  emphasis.)  A  "pr  bablllty 
measure"  ir  "pr  bablllty  distribution"  will  be  a  measure  of 
total  mass  1.  N  tlce  that  we  d  net  require  measures  to  be 
finite.  A  "slgma-flnlte"  measure  Is  a  measure  such  that  X 
Is  the  union  ol  a  denumerable  number  uf  sets,  each  cf  which 
nas  a  finite  measure. 
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Various  condltlcns  are  known  whic.u  Imply  the  existence 
of  a  probability  measure  Q(E)  which  Is  a  stationary  distribution 
for  the  x^-process;  that  is,  Q  satisfies,  for  every  measurable  E, 

(1.1)  Q(E)  -w/'Q(dx)P(x,E). 

X 

If  Xq  has  this  distribution,  sc  has  xn  for  every  n.  Two  sets 
of  conditions  1'or  tne  existence  of  such  a  prebabllity  measure 
were  given  by  Doeblln.  One  set  Is  discussed  In  Doot  J  ,  pp.  190  ffj. 
A  more  general  set  Is  given  In  [6 j  . 

There  are  many  situations  where  there  Is  no  probability 
measure  satisfying  (l.l),  but  wnere  a  s  lutl  n  can  be  1\  und 
If  Q(X)  -a  Is  ail  wed.  Tne  simplest  example  Is  tne  random 
walk  wnere  xn  tikes  Integer  values,  and  can  Increase  ir  decrease 
by  1,  with  probabilities  1/2  each,  at  each  step.  In  this  case, 
a  Solution  U  (l.l)  can  be  obtained  by  assigning  t  any  sei  of 
Integers  a  Q-measure  equal  t  the  numrer  f  Integers  In  the  set. 

All  Integers  are  "equally  probable." 

In  t.hls  paper  a  solution  of  (l.l)  will  always  mean  a  sigma- 
finite  measure  Q  whlcr.  satisfies  (1.1)  for  every  meaaurafle 
set  E.  Tne  ;rincipal  result,  confined  In  The  rern  i,  Is  tr.at 
a  certain  ty;  e  of  recurrence  c  ndltl  n  on  tr.e  x  -process  Insures 
the  existence  t  f  a  s  lutlon  ol  (l.l)  wnlcr.  Is  unique,  .in  1  wnlch 
assigns  positive  measure  to  certain  "rejurrent"  sets. 

It  Is  helpful  t  Consider  S'  me  kn.wn  results  f  r  tne  case 
where  X  Is  denumei  a  c  ie ,  wit:,  states  designated  ty  letters 
l,J,k,s,...,  whlcn  ore  taken  t  te  Integers  0,1,....  Let 
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P  ,  »  P.  /  be  tr.e  one-step  transition  probability  from  1  to 
1  J  1  v1 

j,  and  let  P.  <  be  tr.e  n— step  transition  probability.  Assume 

*  o 

that  all  states  communicate;  i.e.,  for  each  1,J  there  is  an  n 
3JCh  tr.at  P^  ,n  Is  positive.  Assume  also  that  the  chain  Is 
"recurrent";  that  is,  with  probability  1,  every  state  Is  visited 
Infinitely  often.  (If  this  is  true  for  any  starting  point. 

It  la  true  for  all  starting  points.)  See  Feller  |9»  Ch .  15j 
for  discussion.  Under  tnese  conditions,  It  follows  from  a 
result  of  Doefclln  l5j  that  there  Is  a  set  of  positive  finite 
numters  q,,  J  =  0,.,...,  such  that 


Notice  that  the  limit  In  (i.2j  Is  Independent  of  the  starting 
fosltl  ns  1  in  3  k.  In  case  trie  mean  time  for  retjrn  of  the 
state  ba  k  ti  the  starting  state  Is  finite  (under  the  above 
hypothesis  I  "communication"  this  will  be  tr.e  far  either  all 
j v  r,  stites;  t;  e  quantities  q,  are  a  set  of  stationary  pro- 
t  itl  1 1 1 1  •  s  >r  tne  Markov  chain,  provided  they  are  scaled  so 

trelr  sum  Is  1.  Il  the  mean  times  of  return  are  Infinite 
(stl*l  considering  tr.e  recurrent  case/  the  q,  are  n^t  pro- 
Labilities  since  tr.elr  sum  Is  Infinite.  However,  It  was  sh^wn 
by  Denar.  1  tn  it  even  in  this  case,  tr.e  q,  satisfy  tr.e  equations 
f  stat!  narity  (1-3)#  which  are  of  c  urse  satisfied  also  wnen 
the  q ,  t re  ; r  babllltles: 
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(1.3)  Qj  m  £  Q « P 11 #  1  ■  0,1,2,.... 

l  j.0  J  J1 

Another  proof  of  (1,3)  In  the  non-probabl 11 ty  case  was  given  by 
Chung  [2j  and  furnishes  an  Idea  which  1b  very  helpful  In  the 
present  proof  . 

There  are  several  applications  of  Infinite  stationary 
measures  Q.  In  the  case  where  X  is  denumeratle,  one  can  consider 
Infinitely  many  particles  moving  through  the  states  In  X, 
each  one  moving  Independently  of  the  others  according  to  the 
law  of  the  Markov  chain.  In  such  cases  Derman  showed  [4j 
that  If  initially  the  number  of  particles  In  state  J  has  a 
Poisson  distribution  with  mean  q,,  J  =  0,1,...,  this  same 
distribution  will  be  preserved  as  time  goes  on,  and  related 
results  are  also  given.  A  similar  treatment  for  certain  types 
of  continuous— parameter  processes  has  been  given  by  Doob, 

[7  p.  404]  . 

Another  application  was  discussed  by  Harris  and  Robbins, 

[l4j ,  where  the  application  of  an  ergodlc  theorem  due  to  Hopf, 

[It]  shows  that  If  f  and  g  are  real-valued  functions  such  that 
the  right  sided*  (1.4)  below  Is  defined,  then  assuming  a  suit¬ 
able  recurrence  condition  holds,  It  will  be  true  that  for  almost 
all  starting  states  x^ 

f (x  )  ♦...♦f(x  )  J*  f (x )Q(dx) 

(1.4)  Prob  [  11m  — - - —  - - _  -  1. 

n— W  g(>G)  +  . . -+6(xn)  y*g(x)Q(dx) 

The  statement  "almost  all"  Is  with  respect  to  Q— measure.  In 
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some  cases,  such  as  the  one  discussed  in  the  present  paper, 
the  exceptional  set  Is  empty. 

The  analogue  of  (1.4)  for  discrete  Ma^ko/  chains  was 
proved  by  Chung  without  the  use  of  ergodlc  theory. 

A  question  which  will  be  mentioned  only  briefly  in  this 
paper  Is  whether  the  analogue  of  (1.2)  holds  for  the  more 
general  state  space  X  introduced  above.  That  is,  is  it  true 
that  If  0  <  Q(F )  <  oo  then 
N  N 

(1.5)  I  Pn(x,E)/  £  Pn(y,F)-XJ(E)/(S(F) 

n-1  n-1 

holds  for  all  x  and  y  In  X  and  all  E  in  B?  Conditions  are 
Known  under  which  this  le  true;  however,  they  seem  to  be  less 
general  than  th<  se  under  which  Theorem  1  holds. 

Finally  it  may  be  noted  that  even  in  case  Q(X)  is  finite, 
tne  result  of  the  present  paper  applies  to  certain  cases  where 
one  of  the  results  cf  Doeblln  mentioned  above  Is  not  applicable 
(however,  tne  conclusions  are  weaker  than  they  would  be  if 
Doeblln's  result  could  be  applied)  and  where  Doeblln's  other 
result  seems  difficult  to  verify. 
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2 .  Recurrence  C^naltlon;  the  '’Process  on  A11 

We  consider  Markov  processes  as  defined  In  the  first 
paragraph  of  the  Introduction.  Condition  C:  a  countably 
addlve  sigma— fin1  te  measure  m ( E )  Is  defined  on  sets  E  of  B ; 
m(E)  >  0  Implies 

Prob  (x  €  E  infinitely  often  I  Xq)  -  1 

for  all  starting  points  _ln  X.^- 

Now  let  A  be  a  measurable  set  with  m(A)  >  0.  Let  Xq 
be  any  point  In  A.  Then  almost  ail  sequences  Xq,  x^,...,  will 
have  Infinitely  many  elements  In  A.  Let  yQ  «  xo»^l *^2 ’  * ’ *  * 
be  the  successive  members  of  the  sequence  which  belong  to  A. 

It  can  be  verified  that  the  yn  form  a  Markov  proceed- and  we  can 
write  down  Its  transition  function  PA(x,E)  «  Prob  (y  ^  £E  1  yncrx): 

(2.1)  Pa(x,E)  -  P(x,E)t/^;P(x,dy)P(y,£)-|/^_Al/Jc_AP(x  ,dy  )P(y,dz) . 

•P(ZjC.)+.  •  •  . 

We  shall  refer  to  tne  y^-prccees  as  the  process  on  A„  cr  the  A— process. 

'-Conditional  probabilities  will  usually  be  those  defined 
uniquely  by  the  transition  function. 

I  2 

-*Thls  Intuitively  obvious  statement  perhaps  needs  proof. 

It  can  readily  be  shown  that  the  Jclnt  distribution  of 
y  1  ,y2,  . . . ,y  corresponds  tc  wnat  would  be  obtained  by  Iteration 

of  PA,  for  all  n. 
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3.  Motivation  for  Proof  of  Theorem  1 

It  is  evident  from  (1.2)  that  In  the  discrete  case  the 

quantities  q.  are  proportional  to  the  expected  number  of  times 

J 

the  state  Is  J,  over  a  long  time  Interval.  It  follows  from  a 
proof  of  (1.2)  by  Chung  [2]  that  the  ratio  Qj/q^  1b  equal  to 
the  expected  number  of  visits  to  J  between  two  visits  to  1, 
where  1  4  J •  Hence  we  might  think  of  taking  Q(E)  proportional 
to  the  expected  number  of  visits  to  the  set  E  by  a  "particle" 
which  starts  at  some  point  x,  before  returning  to  x.  The 
difficulty  is  that  In  general  the  particle  does  not  have  pro¬ 
bability  1  of  returning  to  the  state  x.  It  then  seems  natural 
to  take  some  reference  set  A,  with  m(A)  >  0,  let  the  particle 
have  some  Initial  probability  distribution  on  A,  and  take 
Q(t)  to  be  the  expected  number  of  visits  to  E  >  “»fore  returning 
to  A.  In  general,  there  would  be  difficulties  with  this  procedure 
but  the  situation  becomes  simple  If  there  is  a  stationery 
probability  distribution  for  the  A— process;  If  the  particle 
has  this  distribution  Initially,  it  will  have  the  same  distri¬ 
bution  each  time  It  returns.  This  type  of  argument  suggests 
the  reason  for  Lemma  1  below.  See  also  Levy,  [18] ,  where  the 
A-prccess  Is  ured  for  discrete  X,  and  Harris,  [12] .  The  proce¬ 
dure  used  by  Halmoe,  [li]  In  going  from  "bounded"  to  "sigma— 

*1 

bounded  spaces  Is  related. 
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4 .  Proof  of  Existence  of  Stationary  Measure 

Theorem  1.  Assume  the  Markov  process  satisfies  the  conditions 

of  paragraph  1  cf  the  Introduction,  and  that  Condition  C 

holds.  Then  (l.l)  has  a  solution  Q  which  is  unique  up  to  a 
- - - 

constant  positive  multiplier,  and  Is  stronger  than  m.  (The 
precise  meaning  of  "solution"  Is  given  in  the  Introduction). 

A  series  of  lemmas  will  be  needed  for  the  proof.  The 

1 

assumptions  of  Theorem  1  will  be  made  for  each  of  t.nem,  although 
Lemma  1  actually  is  true  under  wider  conditions. 

Lemma  1 .  Let  A  be  a  measurable  set  with  0  <  m ( A )  <  oo . 

Suppose  the  "process  on  A"  has  a  stationary  probability  measure 
Qa  satisfying 


(I..1)  «a(E)  QA(dx)PA(x,E) 


for  every  measurable  E  1 n  A .  Then  (l.l;  has  a  solution  Q 
which  assigns  the  same  value  ae  QA  to  subsets  of  A. 

Proof  of  Lemma  1.  The  function  PA(x,E),  which  we  defined 
before  as  the  transition  function  for  the  A — process,  can  be 
defined  by  means  cf  the  right  side  jf  (2.1)  fcr  every  x  In 
X,  and  for  all  measurable  E,  whether  or  not  E  Is  a  subset 
of  A.  The  function  so  defined  is  clearly,  for  each  x,  a 
countably  additive  measure.  An  argument  similar  to  toe  one  we 
shall  use  below  to  snow  that  Q  13  sigma-finite  shows  that  P 

A 

is  a  sigma-finite  measure.  Note  that  if  E  is  a  subset  of  X-A, 
then  Pa(x,E)  Is  the  expected  number  of  visits  to  the  set  E 


-L^That  Is,  m(E)  >  0  implies  Q(E)  >  0. 
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before  a  visit  to  A,  if  the  starting  point  Is  x.  (if  xC  A,  the 
Initial  position  Is  not  counted  as  a  "visit"  to  A.)  Now  let 
A'  denote  the  set  X— A.  Prom  the  definition  of  It  Is  clear 
that  the  following  functional  equation  le  satisfied: 

(4.2)  PA(x,t)  -  P(x,E)  +(/A,  PA(x,dy)P(y,E). 

Motivated  by  the  considerations  Indicated  In  Section  3  above, 
we  now  define,  for  every  measurable  E  In  X, 

(4.?)  Q(K)  Qa  (tlx )  P  (x ,  £) . 

A 

Notice  that  for  Ef  A  ,  Q(e)  «  Qa(E)  Using  (4.1),  (4.2),  and 

(4.3) ,  we  nave 

(-•4  )  Q(dx)P(x,i)-/A  QA(dx)P(x,E)*(ylA,  IJl  QA(dx)PA(x,dyjP(y,E). 

I./A  QA(dx)  lP(x,:-:)t/A.  PA(x,dy)P(y,E)j«  QA(dx)PA(x,E)-ft(E)  . 

Hence  Q  satisfies  (l.l),  and  Is  clearly  a  countably  additive 
measure . 

To  shew  that  Q  le  n  lgma-f  1  nl  te ,  we  note  that  Qa(a)  "  Q(A)  -  1, 
s v.  It  la  sufficient  t~  sh  w  that  A'  la  the  union  cf  a  denumerable 
number  of  sets,  each  with*  finite  Q—  measure.  Define  S,  ,  for 

•  J 

every  pair  f  positive  Integers  1  and  J  by 

(4.0)  S  t  -  |x:x(.A'pF'(x,A)  >  1/  j  ,  1  ,  /  -  1,2,... 


I 

1 
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Prom  Condition  C  It  fellows  that  the  union  of  the  S. ,  contains 

^  v 

A ' .  Since  , C  A',  Q(StJ)  1.  Just  the  expected  number  of  visits 
to  S^j  between  visits  to  A,  If  tfe  ‘‘process  on  A"  has  the 
stationary  probability  distribution  Q^.  It  Is  therefore 
clear  that  Q ( ^ )  Is  finite. 

It  remains  only  to  show  that  Q  Is  stronger  than  m.  Since 
Is  by  assumption  stronger  than  m  on  A,  and  Q  coincides  with 
on  A,  It  will  suffice  to  show  that  Q  Is  stronger  than  m 
on  A'.  Hence  let  £  be  a  measurable  subset  of  A',  with  Q(Ej  *  0. 
Suppose  the  x^-process,  n  >  0,  hae  the  Initial  distribution 
Qa  on  A,  Xq  being  assigned  probability  0  of  being  In  A'. 

Since  Q(E)  1b  the  expected  number  of  visits  to  E  between  vlBlts 
to  A,  Q(E)  -  0  evidently  Implies  that  with  probability  1, 

does  net  belong  to  E  for  any  n.  This  contradicts  Condition 
C,  which  Implies  that  for  any  starting  point  x  there  Is  pro¬ 
bability  1  cf  visiting  E  Infinitely  often. 

This  completes  the  proof  of  Lemma  1 . 

We  next  consider  tne  absolutely  continuous  parts  (with 
respect  to  the  measure  m)  of  P(x,E)  and  Its  iterates.  For 
each  positive  integer  n,  let 

(4.6)  pn(x,E)  *  y?.  fn(x,y)m(dy)  ♦  ?cn(x,Ej, 

wher*  fn(x,y)  is  thus  the  density,  and  P^n  Is  the  singular 
component.  Our  original  condition  that  tne  field  B  should  be 
separable  Implies  that  the  representation  (4.6)  is  j.ssltle 
with  fr‘  a  function  whlcn  Is  measurable  In  the  pair  (x,y), 
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measurability  being  defined  with  respect  to  the  product  space 
(X,XJ.  See  Doeb,  L7  ,  pp .  1 96  ani  6 16 J  .  We  shall  henceforth 
assume  that  the  densities  satisfy  this  measurability  condition. 

Lemma  2 .  Let  r  he  any  real  number  strictly  between  0 
and  1 .  There  exls t  n  measurable  set  A,  a  positive  number  s , 
and  a  positive  Integer  k ,  such  t.nat  C  <  m  ( A )  <  to  ,  and  for  every 
x  C  A 

(2.4)  m  j  y :  y  t  A ,  f1(x,y)  +  .  ..  ♦  fK(x,y)  >  s  {  >  rm(A). 

Fr  cl'  uf  Lemma  <  .  Since  each  of  the  measures  P^n,  n  >  0,  1b 
singular,  we  car.  find,  for  each  x,  a  measurable  set  S(x)  with 

m  ( S  1  x  )  j  =  w  ,  suer,  that 

(A.-)  P  n(X  -  S  'x))  -  0,  n  «  1,2 . 

K.r  e  iffi  x,  let  T(x)  be  trie  (measurable^  x-set  ieflned  by 

{li  ■  T  ( x  )  -  ;  y:l'n(x,y)  «  0,  n  -  1,2 . <  • 

Then  If  x  •  x,  the  probatlllty  Is  1  tr  at.  tr.ere  Is  no  n  such 
tn.it  belongs  t<.  T(x)  -  T(xjS(x).  Hence  fr-m  Condition 

C  ,  r.  (  r  ( x  j  )  -  C  . 

Now  let  ni  be  any  meas jrat  le  set  such  that  0  <  m(Aj  <  a  . 
Kcr  eacr.  xCAlt  leflne  tr.e  measurable  set  «  A  ^(x)  for 

*  *  1 » 2 .  t  y 

(4 . id)  A, , (x)  -  -  y :y  t A, ,  t  1 [x , y)  <-  ...  +f1(x,y)>  1/1  j  . 


1 
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The  remark  above  that  m(T(x))  =  0  shows  that 

(4.11)  mfAj  -  ^,1AU)  =  0. 

Hence  for  each  x  1  there  Is  a  smallest  positive  Integer  1  ■  l(x) 
such  that 

(4.12)  m(An)  >  (  ^  )  m^). 

Now  define 

(4.13)  K,  -  \x:xeA  1  (x )  -  '  j  ,  J  -  1,2,... 

J  ■  A  > 

Then  eac>  x  In  A^  reKngs  to  some  K,.  Hence  we  can  find  an 


Integer  1  such 

tna 

r 

( -*  •  1 4  )  m  ( K  i 

4 

.  .  4 

K  )  >  ?m(AA 

)/(*  r) 

• 

New  define  A  - 

h 

4  .  .  . 

-4  K  .  Then 
i 

f  r  every  x  In 

A  the  rein 

t  Ion 
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f w r  the  1  rocess  cn  A"  have  the  decomposition  (satisfying 
tr.e  neasurcbl  UUL  condition  nentluned  In  connection  with  ( 4 . 6 ) ) 

(*•16)  PAn(x,t)-/^  fAn(x,y)  m(dy )4pA^n(x,E) ,n-l,2, . . . ,x  t A,  EC  A. 

Then  (*.7)  still  hclds,  with  the  same  k  and  s,  l_f  ♦  ...  > 

1?  replaced  by  f*  4-  ...  4  f .  * 

Pro  f  of  Lemma  3-  We  observe  that,  f~r  Ec  A,  ?^(x,F.)  ♦ 

...  *-  Pn(x,K)  Is  the  expected  number  of  visits  to  E  In  n 
steps,  while  P.^(x.E)  4  ...  4  P.n(x,E)  Is  the  expected  number 

/\  n 

of  visits  t  r  In  the  first  n  visits  tc  A.  Hence  clearly  fcr 
every  x  t  A  and  m  A 

(-.17)  PA 1  ♦  ...  4  p  n  >  p 1  4  ...  4  pn ,  n  -  1,2,... 

Le*  A.,  be  a  fut  set  f  A  with  x  (Ap)»  m(A),  sue  a  that 

P.C"(*.A?)  ■  0,  n  -  1,?.... 

Then  for  c  A  ( A  ^  n  »y  defend  cn  x) 

4. 

(•••19)  c/ v  ( :  A*  (x  ,y  )  ►  •  •  •  ■'■f  Ar‘(x  ,y  )  )m(dy  )>y?  (f 1  (x,y  )♦  .  .  .+fn(x,y  )  )m(dy  )  . 

;.en  o  1  r  e  icn  x,  the  rel.atl  r.  f.*  4  ...  *•  f.n  >  f*  4  . . .  4  fn 

M  — 

must  h  Id  f  r  :1m  st  ill  y  Ir.  «  (m— measure.)  Lemma  3  fellows 
Immediately  fr  m  this  fact. 

Lemma  4.  Let  P(x,E)  te  the  Mark~v  transition  function  defined  by 
(a.po)  H (x , h) » tFA1 (x ,E) 4 . . . +PAk (x ,E)j/k ,x £  A ,  E  C A , 


P-720 

-14- 


where  2,  P^(x,r.)t  m_i  k  are  tne  q ..-mt  1 t 1  es  ie fined  In  Levins 
?  and  3  r  r  5  re  r  >  1/  l/? .  Le  t  Pr‘,n  -  1,2,...,  t  e  toe 

1  te  r  j  teg f  P.  fnen  t  ere  1b  a  statl  n  ry  ;  r  ha b  1  i  *  t  y  d  1  v  t  r  1  - 

.  • 

but  1  or.  Qa  ( h  j  f r  P,  e  ■  1 1  s  f’y  1  ,  f  r  all  Tie  .  s jrut le  2mA, 


(4.n)  qa(l)  -,Jl  tiA  ( dx,  h  ;  x , ) . 

Mere  ver  there  Is  a  number  t ,  C  <  t  o  1  ,  e  >cr,  tna t  1  £  >  ( K )  1  s  any 
probability  measure _ n  A , 


(4.22/  .Q.Uj-y;  }  (dx  )Fn  (x  ,r.)  c  tr‘  *,  n  =  1,2,.... 

The  proof  cf  Lemma  s  foil  we  f  r^m  result*  i  f  Doetlin; 

Hee  Doob,  l7  ,  p  p.  1  ff.,  H  we  ver ,  the  transit!  n  function 

h  satisfies  such  a  strong  positivity  r.dltlun  that  a  very 

simple  proof  ^.f  Lemma  4  can  te  plven.  Su:r.  a  proof  is  elver 

,  4 

In  the  Afjendlx  t  tie  present  piper.— 

Lemma  The  dlstrlf  at!  n  Q.  (h)  of  Lemma  '*  is  stall  nary 


f_r 

P  A  ( x  ,  K ) 

Pr  f 

;  tna t  Is,  ( 4 

.  1 )  Is  t>  ■ . 1 1  sf  1  ed 

f  Lemma  P . 

Let  us  detlne  , 

my 

1 robat i 

1 1 1  y  measure 

\  ief  1  ne  1  r.  A  , 

(4.2h) 


J: 


/A 


I  (  dx  )  F  .  (x,t);Vp(L).  y.  j(Jx)  fi[x,z) 


I  4 

-  I!'  b  et  1  in’s 
suf; 1-1 ent  t  take 


result  Is  t  Le^.eei  olrectly, 
r  *r re  iter  than  ^  in  Lemma  s. 
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ll)  and  V4  are  clearly  themselves  probability  measures.  The 
ntf  power  if  the  operator  U  (or  V)  Is  obtained  by  replacing 
PA  by  f^n  (or  R  by  R11  ).  Note  ala„  that  because  of  the  definition 
f  R  (see  (4.?0)),  we  have 

(4 . ?*  j  V  -  (U  +  U2  ♦  .  .  .  *  UN/* 

From  (4.24)  we  see  that  U  and  V  commute:  UnV/T1  -  V^l/1  for  any 
l  sltive  Integers  m  and  n.  N  ;w  (4.?l),  in  the  present  language, 

becomes 


)  Qa  -  vqa, 

•inn  (4.3?)  states  that  for  any  probability  measure  <)  defined 
on  t 

,Vn0(E)  -  QA  ( E )  |  <  tn~\  n  -  1,2, - 

'Jslr.g  (4.2‘^j,  (4.26J,  ana  commutativity,  we  have  (the  measures 
In  (-*..7)  are  evaluated  f  r  s^me  fixed  measurable  set  E) 

{*..1,  UQa  -  uv"qa  -  Vn(UQA)  -  qa  ♦  er 

where  l' f  la  a  quantity  which  Is  less  than  tn  ^  In  magnitude. 

This  -nan  r.ly  tc  true  If  UQ.  •»  Q,  ,  which  Is  the  desired  result. 

M  M 

In  order  to  apj  ly  Lemma  1,  we  need  tc  know  that  Is 
strcnver  than  m  on  A.  Suppose  that  E  Is  a  measurable  buoset 
cf  A  for  which  Qa(E)  -  0.  Con  *der  the  "process  on  A"  yg»y^»-*-» 
where  y.  has  the  distribution  Q^.  Because  of  :he  statlonarlty 
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of  the  y^-process,  we  have 

(4.28)  Prob  (yn€r:)  -  Qa(E)  =  Cf  n  -  1,2,.... 

Because  of  Condition  C,  (4.28)  Implies  m(t)  -  t) . 

We  can  new  apply  Lemma  1.  There  Is  then  a  solution  Q 
ol  (1.1);  we  recall  that  a  solution  Is  a  countably  additive, 

8 lgma-f ini te  measure;  Lemma  1  shows  It  to  be  stronger  than  m. 

It  remains  to  preve  uniqueness.  We  shall  show  that,  except 
f^r  a  multiplicative  constant,  tnere  Is  tnly  one  sigma-flnlte 
measure  which  satisfies  (1.1),  even  among  measures  which  are 
not  necessarily  stronger  than  m.  It  Is  convenient  to  defer 
the  proof  of  uniqueness  to  the  next  section. 


P-728 

&-31-1955 

-17- 


5 •  Uniqueness  and  Related  Toplca 

Throughout  Section  5  we  shall  still  assume  that  the  conditions 
of  the  first  paragraph  of  the  Introduction  hold,  and  that 
Condition  C  Is  satisfied.  Furthermore,  It  will  be  assumed 
that  a  solution  Q(E)  of  (l.l)  hae  been  constructed  by  use  of 
acme  fixed  set  A,  as  In  Lemmas  1-5;  ti.ls  solution  has  been  shown 
tc  be  slgma-finlte  and  stronger  tnan  m,  and  we  shall  reserve 
the  notation  Q  for  It  throughout  the  present  section. 

Lemma  6.  Let  D  be  a  measurable  set  with  0  <  m(D),  Q(D)  <  ao  . 
Then  Q(E)/Q(D)  1b  a  stationary  probability  measure  for  the 
"process  on  D",  satisfying  (P^  is  the  transition  function  fur 
the  recess ) 

(5-1)  Q(E)/Q(D)  -,/^[Q(dx)/Q(D)]PD(x,E),  EoD. 

It  1 8  the  only  stationary  probability  measure  for  tnls  process. 

We  mention  explicitly  tr.at  Q  was  constructed  with  reference 
tc  a  fixed  set  A,  nut  with  reference  to  D. 

Proof  of  Lemma  6.  For  simplicity,  take  Q(D)  ■  1.  Let 
D'  =»  X-D.  Then,  taking  (l.l)  as  our  starting  point  we  obtain 

(5  2)  «(E)  Q(d*)  P(*,E)  .yrD  *  .  ECD. 

Tne  second  term  on  the  right  side  of  (5-?)  can  be  written  ae 


(c-3) 


vy  p.Q(dy  )P(y  ,E)  -c/^f  [y^Q(dx)  P(x,dy)]  P(y,E) 

Jx  ^dx) y)  P(y»£)  m  J  d c/d'  * d1 


Hence  we  have 
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(5.“)  «(E}.^Q(d.K)i:  P(*,F)  +  i/p,P(x,dy)P(y,E)j<-^, 

or 

(5-5)  «(E)><4  «(dx)  [  P(x,E)  +  </^,P(x,dy)P(y,E)j  . 

This  process  can  bs  continued,  replacing  Q(dx)  In  the  last 
term  In  (5-4)  by 

(5-6)  Q(dy)P(y,dx) . 

Hence  me  obtain 

(5.7)  Q(E)>i/^,  9(dx)  [  P(x, £)  +  (/£, P(x,dy)P(y,E)  +  ...j 

Q(dx)  Pd(x,E),  ECD. 

Now  Q(E)  Is  a  probability  measure  on  D  (we  have  taken  Q(D)  -  l). 
Since  Pp  Is  a  Markov  transition  probability  function,  the  right 
side  of  (5-7)  Is  likewise  a  probability  measure  on  D.  Hence 
(5-7)  can  only  be  true  If  the  equality  (5-1)  holds. 

At  this  point  we  observe  that  the  only  property  of  Q  of 
which  we  have  made  use  Is  the  fact  that  It  Is  a  solution  of 
(l.l).  The  above  argument  will  then  go  through  for  any  measurable 
set  D  such  that  0  <  Q(D)  <  cd  ,  and  such  that  Prob  (xn£D 
Inf.  often  J  Xq)  -  1  for  all  Xq. 

Now  consider  a  "process  on  D"  with  variables  z ,  n  -  0,.,..., 
where  for  each  n,  zn  has  the  stationary  distribution  Q  on  D. 
Suppose  D  Is  the  union  of  two  disjoint  nonempty  measurable  sets 
Since  ra(D)  >  0,  either  ^r  D?  must  have  positive 
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m-measure.  It  follows  that  the  two  statements  In  (5*8)  cannot 
8lmul taneoualy  be  true: 

(5.8)  Prob  (zn€D1,  n-1,2, . . .  |ZQ-z)-l,  all 

Prob  (zn€D2#  n-l,2f ..  .  |Zq-z)«1,  all  z  £  . 

The  fact  that  two  such  statements  cannot  be  made  Is  stronger 
than  mere  ergodiclty  and  In  conjunction  with  the  fact  that  Q 
is  stronger  than  m,  implies  the  following  proposition.  Let 
E  be  a  measurable  subset  of  D  and  let  g(z)  be  the  characteristic 
function  of  E.  Then  for  all  (not  just  almost  all)  z  in  D 
we  have 


(5-9) 


Prcb 


n--*ac  n 


Q(E)  | Zq  -  z 


i . 


(See  the  discussion  of  a  similar  situation  in  Doob,  [7 
Theorem  6.2,  p.  220}  ).  Now  any  stationary  probability  distri¬ 
bution  Q'  defined  on  D  for  the  D-procesa  will  correspond  to  an 

15 

ergodlc  stationary  process  zn,  where  zn  has  the  distribution 

Q'  .  The  ergodlc  theorer  of  Blrkhoff  then  implies  that  (5-9) 
holds,  with  Q(E)  replaced  by  Q' (E)  for  almost  all  (Q'— measure ) 
z  in  D.  (We  cannot  say  all  z  at  this  point  since  a  priori 
we  do  not  know  that  Z‘  is  stronger  than  m.)  This  is  a  contra¬ 
diction  unless  Q'  and  Q  are  equal.  Hence  Q  is  unique  among 


^Ergodiclty  follows  as  for  the  Q-process. 
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statlonary  probability  measures  for  the  D-process .  This  completes 
the  proof  of  Lemma  6. 

We  can  get  some  additional  Information  from  (5*9) • 

We  know  tnat  Q  Is  stronger  than  m,  and  It  may  be  strictly 
stronger;  that  Is,  there  may  be  sets  E  with  m  ( E )  ■■  0,  Q(e)  >  0. 

It  can  be  shown  from  (5*9)  that  Q(E)  >  0  Implies 

(5-10)  Prob  (xn^E  l.o.  !Xq)  ■  1,  all  f  X  . 

We  can  now  conclude  the  uniqueness  prucf  for  the  solution 
of  (1.1).  Suppose  Q  Is  our  solution  and  that  Is  any  different 
solution.  Then  there  must  be  a  set  D  such  that 

(5.11)  0  <  m (D ) ,  Q(D) ,  Q:(D)  <  ®  , 

(for  convenience  let  us  take  Q(D)  -  Q^(D)  -  l)  and  such  that  for 
some  subset  E  of  D,  we  have  Q(E)  t  Q^(E).  From  the  discussion 
above  It  follows  that  both  Q  and  must  be  stationary  probability 
distributions  on  D  for  the  D— process;  tne  fact  that  Q  Is  such 
a  distribution  was  proved;  the  fact  that  Qj  is  such  a  distri¬ 
bution  follows  from  (5. 10)  and  the  remark  made  In  tne  course 
of  the  proof  of  Lemma  6,  to  the  effect  that  any  solution  of 
(l.l)  provides  a  stationary  distribution  for  the  D-prucess 
under  the  indicated  conditions.  Since  from  Lemma  6  the  station¬ 
ary  distribution  Is  unique,  we  have  a  contradiction. 

This  completes  tne  proof  of  Thecrem  1. 


6. 


Remarks 
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Theorem  1  la  sometimes  helpful  In  the  finite  case,  Q(X)  <  od 
It  1b  useful  to  knew  in  advance  whether  the  finite  or  Infinite 
case  prevails.  The  fallowing  criterion  will  be  stated  without 
proof.  More  general  ones  could  clearly  be  found. 

Suppose  the  conditions  of  Theorem  1  are  satisfied,  and 
suppose  we  can  find  a  set  A,  0  <  m(A)  <  ct  ,  and  a  positive 
number  d  such  that  the  density  f(x,y)  satisfies 

(6.1)  f  (x,y )  >  d,  x  €  A,  y  e  A. 

p  p 

Let  be  the  recurrence  time  to  A;  that  is,  is  the  smallest 

positive  Integer  such  tnat  Xp€  A.  Then  if  |xQ  -  x)  1b 

bounded  Independently  of  x  1  or  all  xt_A,  we  have  the  finite 
case . 

Although  s.-me  conditions  have  been  found  under  which 
(1.5)  is  true,  (see  [l}j ,  the  author  has  not  been  able  to  prove 
this  under  the  conditions  of  Theorem  1  alone. 

It  does  not  seem  easy  to  apply  the  general  ergodlc  theorems 
of  Halmos  L10j  and  [llj,  Dowker  l8j  ,  and  Hurewlcz,  [17]  to  the 
present  problem.  See  also  Hupf,  [l6j  .  However,  it  would  be 
interesting  to  explore  possible  connections.  As  mentioned 
in  the  introduction,  the  Idea  of  extending  a  finite  measure 
on  a  subset  t(  an  infinite  measure  on  the  whole  space  is  some¬ 
what  related  .  a  procedure  used  by  Halmos,  [.  lj . 

Some  recent  results  of  Edward  Nelson,  in  a  thesis  at 
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the  University  of  Chicago,  may  have  ccnnectlons  with  the 
present  problem.  The  author  has  as  yet  been  unable  to  see 


this  work. 
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Ccnslder  R(x,E)  defined  by  (4.20),  and  let  g(x,y)  be  Its 
density  with  respect  to  m.  Then  there  Is  a  positive  number, 
say  b,  suer,  that  for  all  x€  A, 

(A.l)  m  f  y :  g  ( x  ,  y )  >  bj  >  rm(A)  >  m(A)/i/T, 

where  r  la  the  number  of  Lemma  2,  taken  here  to  be  >  \//2. 
w  define 


(A. 2)  M^E)  -  aup  (Rn(x,E)  -  Rn(y,E)) 

*.y 


Wn  -  ((u,v)  :Fn(u,E)  -  Ftn(v,E)  >  o| 


where  all  p  lnt.9  In  the  definitions  are  In  A. 
let  Aa  denote  the  product  space  (A, A),  and  let  ma 
pr  duct  measure  In  this  space  constructed  from  m. 
be  tr.e  subset  of  Aa  defined  by 


Further  ra  ore, 
denote  the 
Let  Q(x,y ) 


(A.)j 


G(x,y)  -  f(u,v) :g(x,u)g(y,v)  > 


b'i 


2  2  2 

Tnen  ma(G)  >  r  m2(A2)  -  r  (m(A))  .  New  for  each  pair  (u,v)  where 
the  expression  defining  W  Is  positive,  there  is  a  corresponding 
pair  (v,u)  where  the  expressl  n  Is  negative,  and  since  the 
measure  Is  product  measure,  these  two  mutually— exclusive  sets 
have  equal  m^-measures.  Hence  m0(A0-W^)  >  ^  m^(A^).  Hence 

(A.  4)  mp  L  G(A2-Wn)]  >  (r?  -  ^)m2(^). 


With  these  preliminaries  we  can  write 
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(A.5)  Hn+i(x,K)  -  hnfl(y,K)  -C/^R (x,du)Rn(u,E)  - 

/AR(y,dv)Hn(v,H)  «^R(x,du)R(y,dv)(Rn(u,E)-Rn(v,E))< 

A  A 

J w  R  ( x  , d u ) R ( y , d  v ) ( Rn ( u , E ) -Rn ( v  ,  E ) )  < 
n 

Mn(E)//w  R(x,du)R(y,dv) . 
n 

Hence 

(A. 6)  M  ^(E)  <  Mr(hj  8Uf)  i/'Vw  R(x,du)R(y,dv) 

x ,  y  n 

Nuw 

(A. 7)  jZ/l  w  R(x>du)R(y,dv)  > 

A2  n 

'jZ/'.  u  g(x,u)g(y,v]m(dujm(dv j  > 
a2  n 

bc  (r2-  i)  m p  (^)  =  c  >  0 . 

Note  also  that  tne  double  Integral  _n  the  right  side  f  ( A .6  j 
would  be  1  if  it  were  taken  over  tne  whole  space  A2.  Hence,  fr  rr. 
( A . 7 )  we  have 

( A  .  c  J  sup  jZ  R  (x  ,  du  }h  {  y  dvj  <  1  c,  0  <  1  -  <•  <  1. 

x,y  n 

It  f c 1  Iowa  that 

(A.yj  Mn(E)  <  (l-c)n_i,  n  -  1,2, - 
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The  same  type  of  argument  shows  that  the  absolute  value  of  the 
lnflmum  on  x  and  y  of  the  quantity  Rn(x,L)  -  Rn(y,E)  satisfies 
the  Inequality  In  (A. 9).  The  desired  result  follows  readily 


from  this. 


■ 
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